Haptic feedback for touchpads and other touch controls

ABSTRACT

A haptic feedback planar touch control used to provide input to a computer. A touch input device includes a planar touch surface that inputs a position signal to a processor of the computer based on a location of user contact on the touch surface. The computer can position a cursor in a displayed graphical environment based at least in part on the position signal, or perform a different function. At least one actuator is also coupled to the touch input device and outputs a force to provide a haptic sensation to the user contacting the touch surface. The touch input device can be a touchpad separate from the computer&#39;s display screen, or can be a touch screen. Output haptic sensations on the touch input device can include pulses, vibrations, and spatial textures. The touch input device can include multiple different regions to control different computer functions.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.11/805,609 entitled “Haptic Feedback for Touchpads and Other TouchControls,” filed May 23, 2007, which is a continuation of U.S. Pat. No.7,148,875, entitled “Haptic Feedback for Touchpads and Other TouchControls,” issued Dec. 12, 2006, which is a continuation of U.S. Pat.No. 6,429,846, entitled “Haptic Feedback for Touchpads and Other TouchControls,” issued Aug. 6, 2002, which is a continuation-in-part of U.S.Pat. No. 6,563,487, entitled “Haptic Feedback for Directional ControlPads,” issued May 13, 2003, which is a continuation-in-part of U.S. Pat.No. 6,243,078, entitled “Low Cost Force Feedback Pointing Device,”issued Jun. 5, 2001, which is a continuation-in-part of U.S. Pat. No.6,184,868, entitled “Haptic Feedback Control Devices,” issued Feb. 6,2001, which is a continuation-in-part of U.S. Pat. No. 6,088,019,entitled “Low Cost Force Feedback Device with Actuator for Non-PrimaryAxis,” issued Jul. 11, 2000.

BACKGROUND

The subject matter described relates generally to the interfacing withcomputer and mechanical devices by a user, and more particularly todevices used to interface with computer systems and electronic devicesand which provide haptic feedback to the user.

Humans interface with electronic and mechanical devices in a variety ofapplications, and the need for a more natural, easy-to-use, andinformative interface is a constant concern. In the context, humansinterface with computer devices for a variety of applications. One suchapplication is interacting with computer-generated environments such asgames, simulations, and application programs. Computer input devicessuch as mice and trackballs are often used to control a cursor within agraphical environment and provide input in these applications.

In some interface devices, force feedback or tactile feedback is alsoprovided to the user, collectively known herein as “haptic feedback.”For example, haptic versions of joysticks, mice, gamepads, steeringwheels, or other types of devices can output forces to the user based onevents or interactions occurring within the graphical environment, suchas in a game or other application program.

In portable computer or electronic devices, such as laptop computers,mice typically too large a workspace to be practical. As a result, morecompact devices such as trackballs are often used. A more popular devicefor portable computers are “touchpads,” which are small rectangular,planar pads provided near the keyboard of the computer. The touchpadssenses the location of a pointing object by any of a variety of sensingtechnologies, such as capacitve sensors or pressure sensors that detectpressure applied to the touchpad. The user contacts the touchpad mostcommonly with a fingertip and moves his or her finger on the pad to movea cursor displayed in the graphical environment. In other embodiments,the user can operate a stylus in conjunction with the touchpad bypressing the stylus tip on the touchpad and moving the stylus.

One problem with existing touchpads is that there is no haptic feedbackprovided to the user. The user of a touchpad is therefore not able toexperience haptic sensations that assist and inform the user oftargeting and other control tasks within the graphical environment. Thetouchpads of the prior art also cannot take advantage of existinghaptic-enabled software run on the portable computer.

OVERVIEW

An embodiment is directed to a haptic feedback planar touch control usedto provide input to a computer system. The control can be a touchpadprovided on a portable computer, or can be a touch screen found on avariety of devices. The haptic sensations output on the touch controlenhance interactions and manipulations in a displayed graphicalenvironment or when controlling an electronic device.

More specifically, the embodiment relates to a haptic feedback touchcontrol for inputting signals to a computer and for outputting forces toa user of the touch control. The control includes a touch input deviceincluding an approximately planar touch surface operative to input aposition signal to a processor of said computer based on a location ofuser contact on the touch surface. The computer positions a cursor in agraphical environment displayed on a display device based at least inpart on the position signal. At least one actuator is also coupled tothe touch input device and outputs a force on the touch input device toprovide a haptic sensation to the user contacting the touch surface. Theactuator outputs the force based on force information output by theprocessor to the actuator.

The touch input device can be a touchpad separate from a display screenof the computer, or can be included in a display screen of the computeras a touch screen. The touch input device can be integrated in a housingof the computer or handheld device, or provided in a housing that isseparate from the computer. The user contacts the touch surface with afinger, a stylus, or other object. The force is preferably a linearforce output approximately perpendicularly to a plane of the touchsurface of the touch input device, and the actuator can include apiezo-electric actuator, a voice coil actuator, a pager motor, asolenoid, or other type of actuator. In one embodiment, the actuator iscoupled between the touch input device and a grounded surface. Inanother embodiment, the actuator is coupled to an inertial mass, whereinsaid actuator outputs an inertial force on the touch input deviceapproximately along an axis perpendicular to the planar touch surface. Atouch device microprocessor separate from the main processor of thecomputer can receive force information from the host computer andprovide control signals based on the force information to control theactuator.

The haptic sensations, such as a pulse, vibration, or spatial texture,are preferably output in accordance with an interaction of a controlledcursor with a graphical object in the graphical environment. Forexample, a pulse can be output when the cursor is moved between menuelements in a menu, moved over said icon, or moved over a hyperlink. Thetouch input device can include multiple different regions, where atleast one of the regions provides the position signal and at least oneother region provides a signal that is used by the computer to control adifferent function, such as rate control function of a value or a buttonpress. Different regions and borders between regions can be associatedwith different haptic sensations.

An embodiment advantageously provides haptic feedback to a planar touchcontrol device of a computer, such as a touchpad or touch screen. Thehaptic feedback can assist and inform the user of interactions andevents within a graphical user interface or other environment and easecursor targeting tasks. Furthermore, an embodiment allows portablecomputer devices having such touch controls to take advantage ofexisting haptic feedback enabled software. The haptic touch devicesdisclosed herein are also inexpensive, compact and consume low power,allowing them to be easily incorporated into a wide variety of portableand desktop computers and electronic devices.

These and other advantages will become apparent to those skilled in theart upon a reading of the following specification and a study of theseveral figures of the drawing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a haptic touchpad;

FIG. 2 is a perspective view of a remote control device including thetouchpad;

FIG. 3 is a perspective view of a first embodiment of the touchpadincluding one or more actuators coupled to the underside of thetouchpad;

FIG. 4 is a side elevational view of a first embodiment in which apiezo-electric actuator is directly coupled to the touchpad;

FIG. 5 is a side elevational view of a second embodiment of the touchpadincluding a linear actuator;

FIG. 6 is a side elevational view of a third embodiment of the touchpadhaving an inertial mass;

FIG. 7 is a top plan view of an example of a touchpad having differentcontrol regions; and

FIGS. 8 a and 8 b are top plan and side cross sectional views,respectively, of a touch screen embodiment.

DETAILED DESCRIPTION

FIG. 1 is a perspective view of a portable computer 10 including ahaptic touchpad. Computer 10 is preferably a portable or “laptop”computer that can be carried or otherwise transported by the user andmay be powered by batteries or other portable energy source in additionto other more stationary power sources. Computer 10 preferably runs oneor more host application programs with which a user is interacting viaperipherals.

Computer 10 may include the various input and output devices as shown,including a display device 12 for outputting graphical images to theuser, a keyboard 14 for providing character or toggle input from theuser to the computer, and a touchpad 16. Display device 12 can be any ofa variety of types of display devices; flat-panel displays are mostcommon on portable computers. Display device 12 can display a graphicalenvironment 18 based on application programs and/or operating systemsthat are running, such as a graphical user interface (GUI), that caninclude a cursor 20 that can be moved by user input, as well as windows22, icons 24, and other graphical objects well known in GUIenvironments. Other devices may also be incorporated or coupled to thecomputer 10, such as storage devices (hard disk drive, DVD-ROM drive,etc.), network server or clients, game controllers, etc. In alternateembodiments, the computer 10 can take a wide variety of forms, includingcomputing devices that rest on a tabletop or other surface, stand-uparcade game machines, other portable devices or devices worn on theperson, handheld or used with a single hand of the user, etc. Forexample, host computer 10 can be a video game console, personalcomputer, workstation, a television “set top box” or a “networkcomputer”, or other computing or electronic device.

Touchpad device 16 preferably appears externally to be similar to thetouchpads of the prior art. Pad 16 includes a planar, rectangular smoothsurface that can be positioned below the keyboard 14 on the housing ofthe computer 10, as shown, or may be positioned at other areas of thehousing. When the user operates the computer 10, the user mayconveniently place a fingertip or other object on the touchpad 16 andmove the fingertip to correspondingly move cursor 20 in the graphicalenvironment 18.

In operation, the touchpad 16 inputs coordinate data to the mainmicroprocessor(s) of the computer 10 based on the sensed location of anobject on (or near) the touchpad. As with many touchpads of the priorart, touchpad 16 can be capacitive, resistive, or use a different typeof sensing. Some existing touchpad embodiments are disclosed, forexample, in U.S. Pat. No. 5,521,336 and U.S. Pat. No. 5,943,044.Capacitive touchpads typically sense the location of an object on ornear the surface of the touchpad based on capacitive coupling betweencapacitors in the touchpad and the object. Resistive touchpads aretypically pressure-sensitive, detecting the pressure of a finger,stylus, or other object against the pad, where the pressure causesconductive layers, traces, switches, etc. in the pad to electricallyconnect. Some resistive or other types of touchpads can detect theamount of pressure applied by the user and can use the degree ofpressure for proportional or variable input to the computer 10.Resistive touchpads typically are at least partially deformable, so thatwhen a pressure is applied to a particular location, the conductors atthat location are brought into electrical contact. Such deformabilitycan be useful since it can potentially amplify the magnitude of outputforces such as pulses or vibrations on the touchpad. Forces can beamplified if a tuned compliant suspension is provided between anactuator and the object that is moved, as described in U.S. Pat. No.6,680,729. Capacitive touchpads and other types of touchpads that do notrequire significant contact pressure may be better suited in manyembodiments, since excessive pressure on the touchpad may in some casesinterfere with the motion of the touchpad for haptic feedback. Othertypes of sensing technologies can also be used in the touchpad. Herein,the term “touchpad” preferably includes the surface of the touchpad 16as well as any sensing apparatus included in the touchpad unit.

Touchpad 16 preferably operates similarly to existing touchpads, wherethe speed of the fingertip on the touchpad correlates to the distancethat the cursor is moved in the graphical environment. For example, ifthe user moves his or her finger quickly across the pad, the cursor ismoved a greater distance than if the user moves the fingertip moreslowly. If the user's finger reaches the edge of the touchpad before thecursor reaches a desired destination in that direction, then the usercan simply move his or her finger off the touchpad, reposition thefinger away from the edge, and continue moving the cursor. This is an“indexing” function similar to lifting a mouse off a surface to changethe offset between mouse position and cursor. Furthermore, manytouchpads can be provided with particular regions that are each assignedto particular functions that can be unrelated to cursor positioning.Such an embodiment is described in greater detail below with respect toFIG. 7. In some embodiments the touchpad 16 may also allow a user to“tap” the touchpad (rapidly touch and remove the object from the pad) ina particular location to provide a command. For example, the user cantap or “double tap” the pad with a finger while the controlled cursor isover an icon to select that icon.

The touchpad 16 is provided with the ability to output haptic feedbacksuch as tactile sensations to the user who is physically contacting thetouchpad 16. Various embodiments detailing the structure of the hapticfeedback touchpad are described in greater detail below. Preferably, theforces output on the touchpad are linear (or approximately linear) andoriented along the z-axis, approximately perpendicular to the surface ofthe touchpad 16 and the top surface of computer 10. In a differentembodiment, forces can be applied to the touchpad 16 to causeside-to-side (e.g., x-y) motion of the pad in the plane of its surfacein addition to or instead of z-axis motion, although such motion is notpreferred.

Using one or more actuators coupled to the touchpad 16, a variety ofhaptic sensations can be output to the user who is contacting the pad.For example, jolts, vibrations (varying or constant amplitude), andtextures can be output. Forces output on the pad can be at least in partbased on the location of the finger on the pad or the state of acontrolled object in the graphical environment of the host computer 10,and/or independent of finger position or object state. Such forcesoutput on the touchpad 16 are considered “computer-controlled” since amicroprocessor or other electronic controller is controlling themagnitude and/or direction of the force output of the actuator(s) usingelectronic signals. Preferably, the entire pad 16 is provided withhaptic sensations as a single unitary member; in other embodiments,individually-moving portions of the pad can each be provided with itsown haptic feedback actuator and related transmissions so that hapticsensations can be provided for only a particular portion. For example,some embodiments may include a touchpad having different portions thatmay be flexed or otherwise moved with respect to other portions of thepad.

In other embodiments, the touchpad 16 can be provided in a separatehousing that is connected to a port of the computer 10 via a cable orvia wireless transmission and which receives force information from andsends position information to the computer 10. For example, UniversalSerial Bus (USB), Firewire, or a standard serial bus can connect such atouchpad to the computer 10. In such an embodiment, the computer 10 canbe any desktop or stationary computer or device and need not be aportable device.

One or more buttons 26 can also be provided on the housing of thecomputer 10 to be used in conjunction with the touchpad 16. The user'shands have easy access to the buttons, each of which may be pressed bythe user to provide a distinct input signal to the host computer 12.Typically, each button 26 corresponds to a similar button found on amouse input device, so that a left button can be used to select agraphical object (click or double click), a right button can bring up acontext menu, etc. In some embodiments, one or more of the buttons 26can be provided with tactile feedback as described in U.S. Pat. No.6,184,868 and U.S. Pat. No. 6,563,487. Other features of thesedisclosures may also be used.

Furthermore, in some embodiments, one or more moveable portions 28 ofthe housing of the computer device 10 can be included which is contactedby the user when the user operates the touchpad 16 and which can providehaptic feedback. Having a moveable portion of a housing for hapticfeedback is described in U.S. Pat. No. 6,184,868 and U.S. Pat. No.6,088,019. Thus, both the housing can provide haptic feedback (e.g.,through the use of an eccentric rotating mass on a motor coupled to thehousing) and the touchpad 16 can provide separate haptic feedback. Thisallows the host to control two different tactile sensationssimultaneously to the user, for example, a vibration of a low frequencycan be conveyed through the housing to the user and a higher frequencyvibration can be conveyed to the user through the touchpad 16. Eachother button or other control provided with haptic feedback can alsoprovide tactile feedback independently from the other controls.

The host application program(s) and/or operating system preferablydisplays graphical images of the environment on display device 12. Thesoftware and environment running on the host computer 12 may be of awide variety. For example, the host application program can be a wordprocessor, spreadsheet, video or computer game, drawing program,operating system, graphical user interface, simulation, Web page orbrowser that implements HTML or VRML instructions, scientific analysisprogram, virtual reality training program or application, or otherapplication program that utilizes input from the touchpad 16 and outputsforce feedback commands to the touchpad 16. For example, many games andother application programs include force feedback functionality and maycommunicate with the touchpad 16 using a standard protocol/drivers suchas I-Force®, FEELit®, or Touchsense™ available from ImmersionCorporation of San Jose, Calif.

The touchpad 16 can include circuitry necessary to report controlsignals to the microprocessor of the host computer 10 and to processcommand signals from the host's microprocessor. For example, appropriatesensors (and related circuitry) are used to report the position of theuser's finger on the touchpad 16. The touchpad device also includescircuitry that receives signals from the host and outputs tactilesensations in accordance with the host signals using one or moreactuators. In some embodiments, a separate, local microprocessor can beprovided for the touchpad 16 to both report touchpad sensor data to thehost and/or to carry out force commands received from the host, suchcommands including, for example, the type of haptic sensation andparameters describing the commanded haptic sensation. Alternatively, thetouchpad microprocessor can simply pass streamed data from the mainprocessor to the actuators. The term “force information” can includeboth commands/parameters and streamed data. The touchpad microprocessorcan implement haptic sensations independently after receiving a hostcommand by controlling the touchpad actuators; or, the host processorcan maintain a greater degree of control over the haptic sensations bycontrolling the actuators more directly. In other embodiments, logiccircuitry such as state machines provided for the touchpad 16 can handlehaptic sensations as directed by the host main processor. Architecturesand control methods that can be used for reading sensor signals andproviding haptic feedback for a device are described in greater detailin U.S. Pat. No. 5,734,373 and co-pending application nos. 60/156,354,60,133,208, 09/376,649, U.S. Pat. No. 6,639,581 and 60/160,401.

FIG. 2 is a perspective view of another embodiment of a device which caninclude the active touchpad 16. The device can be a handheld remotecontrol device 30, which the user grasps in one hand and manipulatescontrols to access the functions of an electronic device or applianceremotely by a user (such as a television, video cassette recorder or DVDplayer, audio/video receiver, Internet or network computer connected toa television, etc.). For example, several buttons 32 can be included onthe remote control device 30 to manipulate functions of the controlledapparatus. A touchpad 16 can also be provided to allow the user toprovide more sophisticated directional input. For example, a controlledapparatus may have a selection screen in which a cursor may be moved,and the touchpad 16 can be manipulated to control the cursor in twodimensions. The touchpad 16 includes the ability to output hapticsensations to the user as described herein, based on a controlled valueor event. For example, a volume level passing a mid-point or reaching amaximum level can cause a pulse to be output to the touchpad and to theuser.

In one application, the controlled apparatus can be a computer systemsuch as Web-TV from Microsoft Corp. or other computing device whichdisplays a graphical user interface and/or web pages accessed over anetwork such as the Internet. The user can control the direction of thecursor by moving a finger (or other object) on the touchpad 16. Thecursor can be used to select and/or manipulate icons, windows, menuitems, graphical buttons, slider bars, scroll bars, or other graphicalobjects in a graphical user interface or desktop interface. The cursorcan also be used to select and/or manipulate graphical objects on a webpage, such as links, images, buttons, etc. Other force sensationsassociated with graphical objects are described below with reference toFIG. 7.

FIG. 3 is a perspective view of a first embodiment 40 of a touchpad 16for providing haptic feedback to the user. In this embodiment, one ormore piezoelectric actuators 42 are coupled to the underside of thetouchpad 16. The piezoelectric actuator 42 is driven by suitableelectronics, as is well known to those skilled in the art. In oneembodiment, a single piezoelectric actuator 42 is positioned at or nearthe center of the touchpad 16, or off to one side if space constraintsof the housing require such a position. In other embodiments, multiplepiezoelectric actuators 42 can be positioned at different areas of thetouchpad; the dashed lines show one configuration, where an actuator 42is placed at each corner of the pad 16 and at the center of the pad.

The piezoelectric actuators 42 can each output a small pulse, vibration,or texture sensation on the touchpad 16 and to the user if the user iscontacting the touchpad. The entire touchpad 16 is preferably moved withthe forces output by actuator(s) 42. Preferably, the forces output onthe touchpad are linear (or approximately linear) and along the z-axis,approximately perpendicular to the surface of the touchpad 16 and thetop surface of computer 10. In a different embodiment, as mentionedabove, forces can be applied to the touchpad 16 to cause side-to-side(e.g., x-y) motion of the pad in the plane of its surface in addition toor instead of z-axis motion. For example, one linear actuator canprovide motion for the x-axis, and a second linear actuator can providemotion for the y-axis and/or the x-axis.

The frequency of a vibration output by an actuator 42 can be varied byproviding different control signals to an actuator 42. Furthermore, themagnitude of a pulse or vibration can be controlled based on the appliedcontrol signal. If multiple actuators 42 are provided, a strongervibration can be imparted on the touchpad by activating two or moreactuators simultaneously. Furthermore, if an actuator is positioned atan extreme end of the touchpad and is the only actuator that isactivated, the user may experience a stronger vibration on the side ofthe touchpad having the actuator than on the opposite side of thetouchpad. Different magnitudes and localized effects can be obtained byactivating some but not all of the actuators. Since the tip of a user'sfinger that is touching the pad is fairly sensitive, the output forcesdo not have to be of a high magnitude for the haptic sensation to beeffective and compelling.

Besides using a finger to contact the touchpad, the user may also holdother objects that directly contact the touchpad. Any haptic sensationsoutput on the pad can be transmitted through the held object to theuser's hand. For example, the user can hold a stylus having a point thatcontacts the touchpad 16 more precisely than a finger. Other objects mayalso be used. In some embodiments, specialized objects can be used toenhance the haptic sensations. For example, a stylus or other objecthaving a flexible portion or compliance may be able to magnify at leastsome of the touchpad haptic sensations as experienced by the user.

The piezoelectric actuators 42 have several advantages for the touchpad16. These actuators can be made very thin and small, allowing their usein compact housings that are typical for portable electronic devices.They also require very low power, and are thus suitable for devices withlimited power (e.g., powered by batteries). In some embodimentsdescribed herein, power for the actuators can be drawn off a busconnecting the computer to the touchpad (or touch screen). For example,if the touchpad 16 is provided in a separate housing, a Universal SerialBus can connect the pad to the computer and provide power from thecomputer to the pad as well as data (e.g. streaming force data, forcecommands, etc.).

FIG. 4 is a side elevational view of the embodiment 40 of the touchpad16 as shown in FIG. 3. Touchpad 16 is directly coupled to a groundedpiezo-electric actuator 42 which operates to produce a force on thetouchpad 16 when an electrical signal is input to the actuator.Typically, a piezo-electric actuator includes two layers which can moverelative to each other when a current is applied to the actuator; here,the grounded portion of the actuator remains stationary with respect tothe surrounding housing 41 while the moving portion of the actuator andthe touchpad move with respect to the housing 41. The operation ofpiezo-electric actuators to output force based on an input electricalsignal is well known to those skilled the art.

The touchpad 16 can be coupled only to the actuator 42, or can beadditionally coupled to the housing of the computer device at otherlocations besides the actuators 42. Preferably the other couplings arecompliant connections, using a material or element such as a spring orfoam. If such connections are not made compliant, then the touchpad 16itself preferably has some compliance to allow portions of the pad tomove in response to actuator forces and to convey the haptic sensationsto the user more effectively.

Since the touchpad 16 is directly coupled to the actuator 42, anyproduced forces are directly applied to the touchpad 16. The electricsignal preferably is obtained from a microprocessor and any circuitryrequired to convert the microprocessor signal to an appropriate signalfor use with the actuator 42.

FIG. 5 is a side elevational view of another embodiment 50, in which thetouchpad 16 is positioned on one or more springs 52. The springs 52couple the touchpad 16 to the rigid housing of the computer 10 and allowthe touchpad 16 to be moved along the z-axis 56. Only a very small rangeof motion is required to produce effective pulses (olts) or vibrationson the pad 16. Stops (not shown) can be positioned to limit the travelof the touchpad 16 to a desired range along the z-axis.

An actuator 54 is also coupled to the touchpad 16 to impart forces onthe touchpad and cause the touchpad 16 to move along the z-axis. In thepresent embodiment, actuator 54 is a linear voice coil actuator, wherethe moving portion (bobbin) of the actuator is directly coupled to thetouchpad 16. The actuator 54 is grounded to the computer 10 housing andoutputs a linear force on the touchpad 16 and thus drives the touchpadalong the z-axis. A short pulse or jolt can be output, or the movingportion of the actuator can be oscillated to provide a vibration havinga particular desired frequency. The springs 52 cause the touchpad 16 toreturn to a rest position after a force from the actuator causes thetouchpad to move up or down. The springs can also provide a compliantsuspension for the touchpad 16 and allow forces output by the actuator54 to be amplified as explained above. Different types of springelements can be used in other embodiments to couple the touchpad 16 tothe rigid housing, such as leaf springs, foam, flexures, or othercompliant materials.

In some embodiments, the user is able to push the touchpad 16 along thez-axis to provide additional input to the computer 10. For example, asensor can be used to detect the position of the touchpad 16 along thez-axis, such as an optical sensor, magnetic sensor, Polhemus sensor,etc. The position on the z-axis can be used to provide proportionalinput to the computer, for example. In addition, other types of forcescan be output along the z-axis, such as spring forces, damping forces,inertial forces, and other position-based forces, as disclosed in U.S.Pat. No. 6,563,487. In addition, 3-D elevations can be simulated in thegraphical environment by moving the pad to different elevations alongthe z-axis. If the pad 16 can be used as an analog input depending onthe distance the entire pad is moved along the z-axis, and/or ifkinesthetic (force) feedback is applied in the z-axis degree of freedom,then a greater range of motion for the pad 16 along the z-axis isdesirable. An elastomeric layer can be provided if the touchpad 16 isable to be pressed by the user to close a switch and provide button orswitch input to the computer 10 (e.g. using contact switches, opticalswitches, or the like). If such z-axis movement of the pad 16 isallowed, it is preferred that the z-axis movement require a relativelylarge amount of force to move the pad at least initially, since suchz-axis movement may not be desired during normal use of the pad by theuser.

The voice coil actuator 54 preferably includes a coil and a magnet,where a current is flowed through the coil and interacts with themagnetic field of the magnet to cause a force on the moving portion ofthe actuator (the coil or the magnet, depending on the implementation),as is well known to those skilled in the art and is described in U.S.Pat. No. 6,184,868. Other types of actuators can also be used, such as astandard speaker, an E-core type actuator (as described in U.S. Pat. No.6,704,001), a solenoid, a pager motor, a DC motor, moving magnetactuator (described in provisional application no. 60/133,208 and U.S.Pat. No. 6,704,001), or other type of actuator. Furthermore, theactuator can be positioned to output linear motion along an axisperpendicular to the z-axis or along another direction different fromthe z-axis (rotary or linear), where a mechanism converts such outputmotion to linear motion along the z-axis as is well known to thoseskilled in the art.

The touchpad 16 can also be integrated with an elastomeric layer and/ora printed circuit board in a sub-assembly, where one or more actuatorsare coupled to the printed circuit board to provide tactile sensationsto the touchpad 16. Helical springs can also be provided to engageelectrical contacts. Or, multiple voice coil actuators can be positionedat different locations under the touchpad 16. These embodiments aredescribed in U.S. Pat. No. 6,563,487. Any of the actuators described inthat patent can also be used.

FIG. 6 is a side elevational view of a third embodiment 60 of the haptictouchpad 16. In this embodiment, the stationary portion of the actuatoris coupled to the touchpad 16, and the moving portion of the actuator iscoupled to an inertial mass to provide inertial haptic sensations.

Touchpad 16 can be compliantly mounted to the rigid housing of thecomputer device similarly to the embodiments described above. Forexample, one or more spring elements 62 can be coupled between thetouchpad and the housing. These springs can be helical or leaf springs,a compliant material such as rubber or foam, flexures, etc.

One or more actuators 64 are coupled to the underside of the touchpad16. In the embodiment of FIG. 6, a piezoelectric actuator is shown. Oneportion 66 of each actuator 64 is coupled to the touchpad 16, and theother portion 68 is coupled to a mass 70. Thus, when the portion 68 ismoved relative to the portion 66, the mass 70 is moved with the portion68. The mass 70 can be any suitable object of the desired weight, suchas plastic or metal material. The mass 70 is moved approximately alongthe z-axis and is not coupled to the housing, allowing free motion. Themotion of the mass 70 along the z-axis causes an inertial force that istransmitted through the actuator 64 to the touchpad 16, and the touchpad16 moves along the z-axis due to the compliant coupling 62. The motionof the touchpad 16 is felt by the user contacting the touchpad 16 as ahaptic sensation.

In different embodiments, other types of actuators can be used. Forexample, a linear voice coil actuator as described for FIG. 5 can beused, in which an inertial mass is coupled to the linear-moving portionof the voice coil actuator. Other actuators can also be used, such assolenoids, pager motors, moving magnet actuators, E-core actuators, etc.Many actuators used for inertial haptic sensations are described in U.S.Pat. No. 6,211,861. Furthermore, a rotary actuator can be used, wherethe rotary output force is converted to a linear force approximatelyalong the z-axis. For example, the rotary force can be converted using aflexure, as described in U.S. Pat. No. 6,697,043.

In the preferred linear force implementation, the direction or degree offreedom that the force is applied on the touchpad with respect to theinertial mass is important. If a significant component of the force isapplied in the planar workspace of the touchpad (i.e., along the X or Yaxis) with respect to the inertial mass, a short pulse or vibration caninterfere with the user's object motion in one or both of those planardegrees of freedom and thereby impair the user's ability to accuratelyguide a controlled graphical object, such as a cursor, to a giventarget. Since a primary function of the touchpad is accurate targeting,a tactile sensation that distorts or impairs targeting, even mildly, isundesirable. To solve this problem, the touchpad device applies inertialforces substantially along the Z axis, orthogonal to the planar X and Yaxes of the touchpad surface. In such a configuration, tactilesensations can be applied at a perceptually strong level for the userwithout impairing the ability to accurately position a user controlledgraphical object in the X and Y axes of the screen. Furthermore, sincethe tactile sensations are directed in a third degree of freedomrelative to the two-dimensional planar workspace and display screen,jolts or pulses output along the Z axis feel much more likethree-dimensional bumps or divots to the user that come “out” or go“into” the screen, increasing the realism of the tactile sensations andcreating a more compelling interaction. For example, anupwardly-directed pulse that is output when the cursor is moved over awindow border creates the illusion that the user is moving a finger orother object “over” a bump at the window border.

FIG. 7 is a top elevational view of the touchpad 16. Touchpad 16 can insome embodiments be used simply as a positioning device, where theentire area of the pad provides cursor control. In other embodiments,different regions of the pad can be designated for different functions.In some of these region embodiments, each region can be provided with anactuator located under the region, while other region embodiments mayuse a single actuator that imparts forces on the entire pad 16. In theembodiment shown, a central cursor control region 70 is used to positionthe cursor.

The cursor control region 70 of the pad 16 can cause forces to be outputon the pad based on interactions of the controlled cursor with thegraphical environment and/or events in that environment. The user movesa finger or other object within region 70 to correspondingly move thecursor 20. Forces are preferably associated with the interactions of thecursor with displayed graphical objects. For example, a jolt or “pulse”sensation can be output, which is a single impulse of force that quicklyrises to the desired magnitude and then is turned off or quickly decaysback to zero or small magnitude. The touchpad 16 can be jolted in thez-axis to provide the pulse. A vibration sensation can also be output,which is a time-varying force that is typically periodic. The vibrationcan cause the touchpad 16 or portions thereof to oscillate back andforth on the z axis, and can be output by a host or local microprocessorto simulate a particular effect that is occurring in a host application.

Another type of force sensation that can be output on the touchpad 16 isa texture force. This type of force is similar to a pulse force, butdepends on the position of the user's finger on the area of the touchpadand/or on the location of the cursor in the graphical environment. Thus,texture bumps are output depending on whether the cursor has moved overa location of a bump in a graphical object. This type of force isspatially-dependent, i.e. a force is output depending on the location ofthe cursor as it moves over a designated textured area; when the cursoris positioned between “bumps” of the texture, no force is output, andwhen the cursor moves over a bump, a force is output. This can beachieved by host control (e.g., the host sends the pulse signals as thecursor is dragged over the grating). In some embodiments, a separatetouchpad microprocessor can be dedicated for haptic feedback with thetouchpad, and the texture effect and be achieved using local control(e.g., the host sends a high level command with texture parameters andthe sensation is directly controlled by the touchpad processor). Inother cases a texture can be performed by presenting a vibration to auser, the vibration being dependent upon the current velocity of theuser's finger (or other object) on the touchpad. When the finger isstationary, the vibration is deactivated; as the finger is moved faster,the frequency and magnitude of the vibration is increased. Thissensation can be controlled locally by the touchpad processor (ifpresent), or be controlled by the host. Local control by the padprocessor may eliminate communication burden in some embodiments. Otherspatial force sensations can also be output. In addition, any of thedescribed force sensations herein can be output simultaneously orotherwise combined as desired.

Different types of graphical objects can be associated with tactilesensations. Tactile sensations can output on the touchpad 16 based oninteraction between a cursor and a window. For example, a z-axis “bump”or pulse can be output on the touchpad to signal the user of thelocation of the cursor when the cursor is moved over a border of awindow. When the cursor is moved within the window's borders, a textureforce sensation can be output. The texture can be a series of bumps thatare spatially arranged within the area of the window in a predefinedpattern; when the cursor moves over a designated bump area, a bump forceis output on the touchpad. A pulse or bump force can be output when thecursor is moved over a selectable object, such as a link in a displayedweb page or an icon. A vibration can also be output to signify agraphical object which the cursor is currently positioned over.Furthermore, features of a document displaying in a window can also beassociated with force sensations. For example, a pulse can be output onthe touchpad when a page break in a document is scrolled past aparticular area of the window. Page breaks or line breaks in a documentcan similarly be associated with force sensations such as bumps orvibrations.

Furthermore, a menu items in a displayed menu can be selected by theuser after a menu heading or graphical button is selected. Theindividual menu items in the menu can be associated with forces. Forexample, vertical (z-axis) bumps or pulses can be output when the cursoris moved over the border between menu items. The sensations for certainmenu choices can be stronger than others to indicate importance orfrequency of use, i.e., the most used menu choices can be associatedwith higher-magnitude (stronger) pulses than the less used menu choices.Also, currently-disabled menu choices can have a weaker pulse, or nopulse, to indicate that the menu choice is not enabled at that time.Furthermore, when providing tiled menus in which a sub-menu is displayedafter a particular menu element is selected, as in Microsoft Windows™,pulse sensations can be sent when a sub-menu is displayed. This can bevery useful because users may not expect a sub-menu to be displayed whenmoving a cursor on a menu element. Icons can be associated withtextures, pulses, and vibrations similarly to the windows describedabove. Drawing or CAD programs also have many features which can beassociated with similar haptic sensations, such as displayed (orinvisible) grid lines or dots, control points of a drawn object, etc.

In other related interactions, when a rate control or scrolling functionis performed with the touchpad (through use of the cursor), a vibrationcan be displayed on the device to indicate that scrolling is in process.When reaching the end of a numerical range that is being adjusted (suchas volume), a pulse can be output to indicate that the end of the rangehas been reached. Pulse sensations can be used to indicate the locationof the “ticks” for discrete values or settings in the adjusted range. Apulse can also be output to inform the user when the center of the rangeis reached. Different strength pulses can also be used, larger strengthindicating the more important ticks. In other instances, strength and/orfrequency of a vibration can be correlated with the adjustment of acontrol to indicate current magnitude of the volume or other adjustedvalue. In other interactions, a vibration sensation can be used toindicate that a control function is active. Furthermore, in some cases auser performs a function, like selection or cutting or pasting adocument, and there is a delay between the button press that commandsthe function and the execution of the function, due to processing delaysor other delays. A pulse sensation can be used to indicate that thefunction (the cut or paste) has been executed.

Furthermore, the magnitude of output forces on the touchpad can dependon the event or interaction in the graphical environment. For example,the force pulse can be a different magnitude of force depending on thetype of graphical object encountered by the cursor. For example, apulses of higher magnitude can be output when the cursor moves overwindows, while pulses of lower magnitude can be output when the cursormoves over icons. The magnitude of the pulses can also depend on othercharacteristics of graphical objects, such as an active window asdistinguished a background window, file folder icons of differentpriorities designated by the user, icons for games as distinguished fromicons for business applications, different menu items in a drop-downmenu, etc. The user or developer can also preferably associateparticular graphical objects with customized haptic sensations.

User-independent events can also be relayed to the user using hapticsensations on the touchpad. An event occurring within the graphicalenvironment, such as an appointment reminder, receipt of email,explosion in a game, etc., can be signified using a vibration, pulse, orother time-based force. The force sensation can be varied to signifydifferent events of the same type. For example, vibrations of differentfrequency can each be used to differentiate different events ordifferent characteristics of events, such as particular users sendingemail, the priority of an event, or the initiation or conclusion ofparticular tasks (e.g. the downloading of a document or data over anetwork). When the host system is “thinking,” requiring the user to waitwhile a function is being performed or accessed (usually when a timer isdisplayed by the host) it is often a surprise when the function iscomplete. If the user takes his or her eyes off the screen, he or shemay not be aware that the function is complete. A pulse sensation can besent to indicate that the “thinking” is over.

A software designer may want to allow a user to be able to selectoptions or a software function by positioning a cursor over an area onthe screen using the touchpad, but not require pressing a physicalbutton or tapping the touchpad to actually select the option. Currently,it is problematic to allow such selection because a user has physicalconfirmation of execution when pressing a physical button. A pulse sentto the touchpad can act as that physical confirmation without the userhaving to press a button or other control for selection. For example, auser can position a cursor over a web page element, and once the cursoris within the desired region for a given period of time, an associatedfunction can be executed. This is indicated to the user through atactile pulse sent to the pad 16.

The above-described force sensations can also be used in games orsimulations. For example, a vibration can be output when auser-controlled racing car is driving on a dirt shoulder of a displayedroad, a pulse can be output when the car collides with another object,and a varying-frequency vibration can be output when a vehicle enginestarts and rumbles. The magnitude of pulses can be based on the severityof a collision or explosion, the size of the controlled graphical objector entity (and/or the size of a different graphical object/entity thatis interacted with), etc. Force sensations can also be output based onuser-independent events in the game or simulation, such as pulses whenbullets are fired at the user's character.

The above haptic sensations can be similar to those described in U.S.Pat. No. 6,243,078 and U.S. Pat. No. 6,211,861. Other control devices orgrips that can include a touchpad 16 in its housing include a gamepad,mouse or trackball device for manipulating a cursor or other graphicalobjects in a computer-generated environment; or a pressure sphere or thelike. For example, the touchpad 16 can be provided on the housing of acomputer mouse to provide additional input to the host computer.Furthermore, selective disturbance filtering of forces, as described inU.S. Pat. No. 6,020,876, and shaping of force signals to drive thetouchpad with impulse waves as described in U.S. Pat. No. 5,959,613, canbe used. Such impulses are also effective when driven with stored powerin a battery on the computer 10 or from a bus such as USB connected to ahost computer.

The touchpad 16 can also be provided with different control regions thatprovide separate input from the main cursor control region 70. In someembodiments, the different regions can be physically marked with lines,borders, or textures on the surface of the pad 16 (and/or sounds fromthe computer 10) so that the user can visually, audibly, and/or ortactilely tell which region he or she is contacting on the pad.

For example, scroll or rate control regions 62 a and 62 b can be used toprovide input to perform a rate control task, such as scrollingdocuments, adjusting a value (such as audio volume, speaker balance,monitor display brightness, etc.), or panning/tilting the view in a gameor virtual reality simulation. Region 62 a can be used by placing afinger (or other object) within the region, where the upper portion ofthe region will increase the value, scroll up, etc., and the lowerportion of the region will decrease the value, scroll down, etc. Inembodiments that can read the amount of pressure placed on the pad 16,the amount of pressure can directly control the rate of adjustment;e.g., a greater pressure will cause a document to scroll faster. Theregion 62 b can similarly be used for horizontal (left/right) scrollingor rate control adjustment of a different value, view, etc.

Particular haptic effects can be associated with the control regions 62a and 62 b. For example, when using the rate control region 62 a or 62b, a vibration of a particular frequency can be output on the pad 16. Inthose embodiments having multiple actuators, an actuator placed directlyunder the region 62 a or 62 b can be activated to provide a morelocalized tactile sensation for the “active” (currently used) region. Asa portion of a region 62 is pressed for rate control, pulses can beoutput on the pad (or region of the pad) to indicate when a page hasscroll by, a particular value has passed, etc. A vibration can also becontinually output while the user contacts the region 62 a or 62 b.

Other regions 64 can also be positioned on the touchpad 16. For example,each of regions 64 provides a small rectangular area, like a button,which the user can point to in order to initiate a function associatedwith the pointed to region. The regions 64 can initiate such computerfunctions as running a program, opening or closing a window, going“forward” or “back” in a queue of web pages in a web browser, poweringthe computer 10 or initiating a “sleep” mode, checking mail, firing agun in a game, cutting or pasting data from a buffer, selecting a font,etc. The regions 64 can duplicate functions and buttons provided in anapplication program or provide new, different functions.

Similarly to regions 62, the regions 64 an each be associated withhaptic sensations; for example, a region 64 can provide a pulsesensation when it has been selected by the user, providing instantfeedback that the function has been selected. Furthermore, the sametypes of regions can be associated with similar-feeling hapticsensations. For example, each word processor related region 64 can, whenpointed to, cause a pulse of a particular strength, while eachgame-related region can provide a pulse of different strength or avibration. Furthermore, when the user moves the pointing object from oneregion 62 or 64 to another, a haptic sensation (such as a pulse) can beoutput on the pad 16 to signify that a region border has been crossed.

In addition, the regions are preferably programmable in size and shapeas well as in the function with which they are associated. Thus, thefunctions for regions 64 can change based on an active applicationprogram in the graphical environment and/or based on user preferencesinput to and/or stored on the computer 10. Preferably, the size andlocation of each of the regions can be adjusted by the user or by anapplication program, and any or all of the regions can be completelyremoved if desired. Furthermore, the user is preferably able to assignparticular haptic sensations to particular areas or types of areas basedon types of functions associated with those areas, as desired. Differenthaptic sensations can be designed in a tool such as Immersion Studio™available from Immersion Corporation of San Jose, Calif.

It should be noted that the regions 62 and 64 need not be physicalregions of the touchpad 16. That is, the entire touchpad 16 surface needmerely provide coordinates of user contact to the processor of thecomputer and software on the computer can designate where differentregions are located. The computer can interpret the coordinates and,based on the location of the user contact, can interpret the touchpadinput signal as a cursor control signal or a different type of signal,such as rate control, button function, etc. The local touchpadmicroprocessor, if present, may alternatively interpret the functionassociated with the user contact location and report appropriate signalor data to the host processor (such as position coordinates or a buttonsignal), thus keeping the host processor ignorant of the lower levelprocessing. In other embodiments, the touchpad 16 can be physicallydesigned to output different signals to the computer based on differentregions marked on the touchpad surface that are contacted by the user;e.g. each region can be sensed by a different sensor or sensor array.

FIGS. 8 a and 8 b are top plan and side cross-sectional views,respectively, of another computer device embodiment 80 including a formof the haptic touchpad 16. Device 80 is in the form of a portablecomputer device such as “personal digital assistant” (PDA), a“pen-based” computer, “electronic book”, or similar device (collectivelyknown as a “personal digital assistant” or PDA herein). Those deviceswhich allow a user to input information by touching a display screen orreadout in some fashion are primarily relevant to this embodiment. Suchdevices can include the Palm Pilot from 3Com Corp., the Newton fromApple Computer, pocket-sized computer devices from Casio,Hewlett-Packard, or other manufacturers, cellular phones or pagershaving touch screens, etc.

For example, scroll or rate control regions 72 a and 72 b can be used toprovide input to perform a rate control task, such as scrollingdocuments, adjusting a value (such as audio volume, speaker balance,monitor display brightness, etc.), or panning/tilting the view in a gameor virtual reality simulation. Region 72 a can be used by placing afinger (or other object) within the region, where the upper portion ofthe region will increase the value, scroll up, etc., and the lowerportion of the region will decrease the value, scroll down, etc. Inembodiments that can read the amount of pressure placed on the pad 16,the amount of pressure can directly control the rate of adjustment;e.g., a greater pressure will cause a document to scroll faster. Theregion 72 b can similarly be used for horizontal (left/right) scrollingor rate control adjustment of a different value, view, etc.

Particular haptic effects can be associated with the control regions 72a and 72 b. For example, when using the rate control region 72 a or 72b, a vibration of a particular frequency can be output on the pad 16. Inthose embodiments having multiple actuators, an actuator placed directlyunder the region 72 a or 72 b can be activated to provide a morelocalized tactile sensation for the “active” (currently used) region. Asa portion of a region 72 is pressed for rate control, pulses can beoutput on the pad (or region of the pad) to indicate when a page hasscroll by, a particular value has passed, etc. A vibration can also becontinually output while the user contacts the region 72 a or 72 b.

Other regions 74 can also be positioned on the touchpad 16. For example,each of regions 74 provides a small rectangular area, like a button,which the user can point to in order to initiate a function associatedwith the pointed-to region. The regions 74 can initiate such computerfunctions as running a program, opening or closing a window, going“forward” or “back” in a queue of web pages in a web browser, poweringthe computer 10 or initiating a “sleep” mode, checking mail, firing agun in a game, cutting or pasting data from a buffer, selecting a font,etc. The regions 74 can duplicate functions and buttons provided in anapplication program or provide new, different functions.

Similarly to regions 72, the regions 74 an each be associated withhaptic sensations; for example, a region 74 can provide a pulsesensation when it has been selected by the user, providing instantfeedback that the function has been selected. Furthermore, the sametypes of regions can be associated with similar-feeling hapticsensations. For example, each word-processor related region 74 can, whenpointed to, cause a pulse of a particular strength, while eachgame-related region can provide a pulse of different strength or avibration. Furthermore, when the user moves the pointing object from oneregion 72 or 74 to another, a haptic sensation (such as a pulse) can beoutput on the pad 16 to signify that a region border has been crossed.

In the embodiments of touch input devices (touchpad and touch screen)described herein, it is also advantageous that contact of the user isdetected by the touch input device. Since haptic feedback need only beoutput when the user is contacting the touch device, this detectionallows haptic feedback to be stopped (actuators “turned off”) when noobjects are contacting the touch input device. This feature can conservebattery power for portable devices. If a local touch devicemicroprocessor (or similar circuitry) is being used in the computer,such a microprocessor can turn off actuator output when no user contactis sensed, thus alleviating the host processor of additionalcomputational burden.

It should be noted that the regions 72 and 74 need not be physicalregions of the touchpad 16. That is, the entire touchpad 16 surface needmerely provide coordinates of user contact to the processor of thecomputer and software on the computer can designate where differentregions are located. The computer can interpret the coordinates and,based on the location of the user contact, can interpret the touchpadinput signal as a cursor control signal or a different type of signal,such as rate control, button function, etc. The local touchpadmicroprocessor, if present, may alternatively interpret the functionassociated with the user contact location and report appropriate signalor data to the host processor (such as position coordinates or a buttonsignal), thus keeping the host processor ignorant of the lower levelprocessing. In other embodiments, the touchpad 16 can be physicallydesigned to output different signals to the computer based on differentregions marked on the touchpad surface that are contacted by the user;e.g. each region can be sensed by a different sensor or sensor array.

1. A haptic feedback device, comprising: of a user interface deviceconfigured to display a graphical object to a user via a touch screen,wherein the graphical object is programmed to have a first state or asecond state; and an actuator coupled to the touch screen and configuredto impart a haptic effect to the haptic feedback device when a user'sinput on the touch screen selects the graphical object only when in thefirst state, wherein the first state is an active state.
 2. A hapticfeedback device comprising: a housing; one or more portions separatelymovable relative to the housing; and an actuator configured to impart afirst haptic force to one of the separately movable portions and asecond haptic force to another of the separately movable portions, or tothe housing.
 3. A method of scrolling on an electronic device, themethod comprising: selecting a touch screen configured to display agraphical interface having a plurality of graphical objects; sensing auser's input with the touch screen, wherein the user's input imparts ascrolling action among the plurality of graphical objects in thegraphical interface; outputting a first haptic effect upon sensing theuser's input at an edge of a scrolling control region of the touchscreen; and outputting a second haptic effect upon sensing the user'sinput at another location on the scrolling control region.
 4. A hapticfeedback device comprising: a housing; one or more portions separatelymovable relative to the housing; a first actuator configured to imparthaptic force to one of the separately movable portions; and a secondactuator configured to impart haptic force to another of the separatelymovable portions, or to the housing.
 5. A portable communication devicecomprising: a touch screen capable of displaying graphical objectstherethrough; a first actuator configured to impart a first hapticeffect to the touch screen at a first frequency; and a second actuatorconfigured to impart a second haptic effect to the touch screen at asecond frequency simultaneously to provide a third haptic effect to befelt by a user.
 6. A method of indicating relative importance inselection of a displayed graphical object, the method comprising:displaying a plurality of graphical objects through a touch screen,wherein the graphical objects are individually selectable by a user;sensing the user's selection of one or more graphical objects via thetouch screen; outputting a first haptic effect upon sensing a firstgraphical object is selected by the user, wherein the first graphicalobject has a first importance value; and outputting a second hapticeffect upon sensing a second graphical object is selected by the user,wherein the second graphical object has a second importance value;wherein the second haptic effect is greater in magnitude than the firsthaptic effect.